Introduction

Diagnostic and prognostic models are typically evaluated with measures of accuracy that do not address clinical consequences. Decision-analytic techniques allow assessment of clinical outcomes but often require collection of additional information and may be cumbersome to apply to models that yield a continuous result. Decision curve analysis is a method for evaluating and comparing prediction models that incorporates clinical consequences, requires only the data set on which the models are tested, and can be applied to models that have either continuous or dichotomous results. This document will walk you through how to perform a decision curve analysis (DCA) in many settings, and how to interpret the resulting curves. In DCA prediction models are compared to two default strategies: 1) assume that all patients are test positive and therefore treat everyone, or 2) assume that all patients are test negative and offer treatment to no one. “Treatment” is considered in the widest possible sense, not only drugs, radiotherapy or surgery, but advice, further diagnostic procedures or more intensive monitoring. For more details on DCA, visit www.decisioncurveanalysis.org. You’ll find the original articles explaining the details of the DCA derivation along with other papers providing more details. Below we will walk through how to perform DCA for binary and time-to-event outcomes using R, Stata, SAS, and Python. The code is provided for all languages and the results shown are the output from the R DCA functions.

Select a language

R Stata SAS Python

Install & Load

Use the scripts below to install the DCA functions and/or load them for use.

R
# install dcurves from CRAN
install.packages("dcurves")

# load package
library(dcurves)
Stata
* install dca functions from GitHub.com
net install dca, from("https://raw.github.com/ddsjoberg/dca.stata/master/") replace
SAS
/* source the dca macros from GitHub.com */
/* you can also navigate to GitHub.com and save the macros locally */
filename dca url "https://raw.githubusercontent.com/ddsjoberg/dca.sas/main/dca.sas";
filename stdca url "https://raw.githubusercontent.com/ddsjoberg/dca.sas/main/stdca.sas";
%include dca;
%include stdca;

Binary Outcomes

Univariate Decision Curve Analysis

We will be working with the example dataset df_binary included with the package. The dataset includes information on 750 patients who have recently discovered they have a gene mutation that puts them at a higher risk for harboring cancer. Each patient has been biopsied and we know their cancer status. It is known that older patients with a family history of cancer have a higher probability of harboring cancer. A clinical chemist has recently discovered a marker that she believes can distinguish between patients with and without cancer. We wish to assess whether or not the new marker does indeed identify patients with and without cancer. If the marker does indeed predict well, many patients will not need to undergo a painful biopsy.

First, we want to confirm family history of cancer is indeed associated with the biopsy result.

R
# build logistic regression model
mod <- glm(am ~ mpg, data = mtcars, family = binomial)

# model summary
gtsummary::tbl_regression(mod, intercept = TRUE)
Stata
* build logistic regression model
logit am mpg
SAS
/* build logistic regression model */
PROC LOGISTIC DATA = test DESCENDING;
  MODEL am = mpg;
RUN;

Characteristic log(OR)1 95% CI1 p-value
(Intercept) -6.6 -12, -2.8 0.005
mpg 0.31 0.12, 0.59 0.008

1 OR = Odds Ratio, CI = Confidence Interval